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(57) Abstract: Methods of displaying and evaluating of a mo- 
tion data in a motion game apparatus are disclosed. The objec- 
tive of the present invention can be achieved by steps as follow 
and the image method helping a game player follow the motion 
of a dancer easily: The first step in which one calculate the basic 
sensor position that must be detected when a game player fol- 
lows the motion of a dancer, and the second step in which one 
calculates the sensor position of each camera in order to obtain 
the basic sensor position, and the third step in which one estab- 
lishes the target domain of each camera having the particular 
domain limit around the sensor position of each camera, and the 
fourth step in which one judges if the sensor of a game player is 
detected in the target domain. 



WO 02/30535 Al IIIIIIIIIIIBIIIIIIIIIIIIIIII1I1IIIIIIIII1IIIIIIIII 



patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, For two-letter codes and other abbreviations, refer to the "Guid- 

IT, LU, MC. NL, PT, SE, TR). OAPI patent (BF, BJ, CF, ance Notes on Codes and Abbreviations" appearing at the begin- 

CG, CI, CM, GA, GN, GQ, GW, ML, MR, NE, SN, TD, ning of each regular issue of the PCT Gazette. 
TG). 

Published: 

— with international search report 



f 



WO 02/30535 



PCT/KR01/01710 



METHOD OF DISPLAYING AND EVALUATING MOTION DATA USED 

IN MOTION GAME APPARATUS 



The present invention relates to method of playing and evaluating a motion 
data in a motion game apparatus, and more particularly to methods of playing the 
motion data so that a game player can easily follow the motion of an original actor 
and of evaluating the motion of the game player following the motion data. 



Recently, what is called, a DDR game apparatus has been all the fashion. 
It has a music play device and a floor body sensing the foot action of a game 
player. In this DDR game apparatus, the foot action in which the game player 

15 will follows, is informed to the game player by either a monitor or a foot action . 
direction of the floor body together with music played by the music play device, 
and the game player follows the foot action in accordance with timing. 
As an apparatus further advanced than the DDR game apparatus, another game 
apparatus in which a sensor for sensing hand stretching action as well as the foot 

20 action of the game player is installed at a certain position thereof has been 
introduced. It can check the motions of both foot and hand actions using the 
sensor. 

However, there are problems that it is impossible to display natural motion of the 
original actor and to evaluate overall motion of the game player following the 
25 motion, since the above described conventional game apparatuses can only sense 
the foot action or the foot action and hand stretching actions at a designated 
position. 



TECHNICAL FIELD 
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DISCLOSURE OF INVENTION 

It is therefore one object of the present invention to provide a method of 
playing a motion data in which a game player can easily follow the motion of an 
original actor in a motion game apparatus displaying the motion of the original 
5 actor, for solving the above problems. 

Furthermore, it is the other object of the present invention to provide a 
method of evaluating a motion data capable of real-time and rapidly evaluating the 
motion of a game player following the motion of an original actor in a motion 
game apparatus displaying the motion of the original actor. - 

10 The one object of the present invention can be achieved by a method of 

displaying a motion data in a motion game apparatus having information of a basic 
frame displaying main motion of an original actor, and playing the motion data of 
the original actor consisting of a plurality of frames on a display device, 
comprising : a first step of setting play position points Xf, Yf and Zf on the display 

15 device for playing continuous action of the original actor, and initial position 
points Xi, Yi and Zi on the display device for displaying beforehand the basic 
frame which will be played after a predetermined time At; a second step of 
displaying the continuous action of the original actor at the play position points Xf, 
Yf and Zf based on present time t, and simultaneously displaying the basic frame 

20 at the initial position points Xi, Yi and Zi after drawing out the basic frame which 
will be displayed at t + At; and a third step of displaying with gradually moving the 
basic frame displayed at the initial position points Xi, Yi .and Zi to the play 
position points Xf, Yf and Zf as time goes by. 

Furthermore, the one object of the present invention can be achieved by a 

25 method of displaying a motion data in a motion game apparatus having 
information of a basic frame displaying main motion of an original actor, 
consisting of a plurality of frames, having play position points Xf, Yf and Zf on a 
display device for playing continuous action of the original actor and initial 
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position points Xi, Yi and Zi on the display device for displaying beforehand the 
basic frame which will be played after a predetermined time At, and displaying the 
motion data of the original actor on the display device, comprising ; a first step of 
displaying continuously the continuous action of the original actor at the play 

5 position points Xf, Yf and Zf based on a present time t, and simultaneously 
displaying the basic frame at the initial position points Xi, Yi and Zi after drawing 
out the basic frame which will be displayed at t + At; and a second step of 
displaying with gradually moving the basic frame displayed at the initial position 
points Xi, Yi and Zi to the play position points Xf, Yf and Zf as time goes by. 

10 Moreover, the other object of the present invention can be achieved by a 

method of evaluating the motion of a gam<5 player following the motion data of an 
original actor displayed on a the 
f information of a basic frame displaying main motion of the original actor, 
' consisting of a plurality of frames, and setting play position points Xf, Yf and Zf 

15 on the display device for playing continuous action of the original actor and initial 
* position points Xi, Yi and Zi on the display device for displaying beforehand the 
basic frame which will be played after a predetermined time At, comprising ; a 
first step of storing the three-dimensional motion data of the original actor 
retargeted after converting the three-dimensional motion data of the original actor 

20 by reflecting the body size of the game player; a second step of displaying 
continuously the continuous action of the original actor at the play position points 
Xf, Yf and Zf based on present time t using the retargeted three-dimensional 
motion data of the original actor and simultaneously displaying the basic frame at 
the initial position points Xi, Yi and Zi after drawing out the basic frame which 

25 will be displayed at t + At; a third step of displaying with gradually moving the 
basic frame displayed at the initial position points Xi, Yi and Zi to the play 
position points Xf, Yf and Zf as time goes by; a fourth step of extracting and 
storing the three-dimensional motion data of the game player learning the same 
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motion as the original actor with following the motion of the original actor 
displayed; a fifth step of calculating the coordinate value of a reference sensor for 
the original actor corresponding to a sensor position attached to the game player 
from the basic frame displayed at the play position points Xf, Yf and Zf of the 

5 third step; and a sixth step of comparing the three-dimensional motion data of the 
game player with the coordinate value of the reference sensor. . 

The nature, utility, and further features of the present invention will be 
clearly apparent from the following detailed description with respect to preferred 
embodiments of the . invention when read in conjunction with the accompanying 

10 drawings briefly described below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Further objects and other advantages of the present invention will be 
apparent from the following description in conjunction with the attached drawings, 
15 in which: 

FIG. 1 is a configuration diagram of a motion game apparatus of the present 
invention. 

FIG. 2 is a block diagram of a motion game apparatus applied to the 
20 present invention. 

FIG. 3 is a flow chart showing retargeting processing sequence. 

FIG. 4 is a frame structure diagram for describing one example of an image 
frame configuration of a three-dimensional original actor according to the present 
invention. 

25 FIG. 5 and FIG. 6 are description drawings for describing a method of 

displaying preparation action of an original actor. 

FIG. 7 is a flow chart showing overall action displaying and evaluating a 
motion using a motion frame of FIG. 4. 

4 



WO 02/30535 



PCT/KR01/01710 



FIG. 8 is a flow chart describing a method of evaluating a motion of a 
game player. 

FIG. 9a to FIG. 9d are a flow chart showing performance sequence of the 
present invention in detail. 

5 

DETAILED DESCRIPTION OF THE INVENTION 

Preferred embodiments of the present invention will now be described in 
detail with reference to the accompanying drawings. 

Hereinafter, the present invention will be explained in detail with reference 
10 to the accompanying drawings. 

FIG. 1 is a configuration diagram of a motion game apparatus of the 
present invention. The motion game apparatus includes a plurality of cameras 
100, a display device 200 displaying a motion of an original actor, an input section 
receiving input from a game player, and a sound device. The game player 
15 attaches a plurality of optical sensors (not shown) to his body and takes a motion in 
a certain region range capable of be sensed by the camera. The camera 100 
monitors the motion of the game player using the optical sensors attached to the 
body of the game player. 

Preferably, the optical sensors are attached to all articulation parts of the 
20 game player if possible. More preferably, the number of the optical sensors is 
appropriately selected in view of processing rate because the number of image 
processing operations increase as the number of the optical sensors increases. 
Furthermore, while it is possible to obtain precise data as the number of the camera 
increase, it is preferable to appropriately select the number of the cameras in view 
25 of the same problem as the number of the sensors above described. 

FIG. 2 is a block diagram of a motion game apparatus applied to the 
present invention. The motion game apparatus of the present invention includes 
an input section receiving an input from a game player, a control section, a 
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sensor/camera, an image output device, a music D/B, a character/stage DB, a 
sound output device, and an image memory. The control section has a motion 
selection section outputting a motion selection signal in accordance with an output 
of the input section, a motion data storage section storing a plurality of motion data 
5 of an original actor, a retargeting processing section retargeting the motion of the 
original actor, a retargeted original actor motion data storage section storing a 
retargeted motion data, a game player motion data storage section storing the 
motion data of the game player, a motion evaluation section evaluating after 
comparing the retargeted motion data of the original actor with the motion data of 
10 the game player, a motion capture section capturing the motion of the game player, 
and a resultant image output section displaying the output of the motion evaluation 
section on the image output device. 

The operation flow of a motion game apparatus in FIG. 2 is will be now 
explained. 

15 Firstly, the game player selects one of the plurality of motion data using the 

input section. The motion select section selects the one of the plurality of motion 
data stored to the motion data storage section and then inputs the selected data to 
the retargeting processing section. 

The retargeting processing section receives a body size from the game 

20 player or senses the body size by image processing using the camera, extracts the 
body size difference between the game player and the original actor, carries out 
retargeting process for the motion data of the original actor, and stores the 
retargeted motion data of the original actor in the motion data storage section. 
Specific description with respect to the retargeting will be described in FIG. 3 later. 

25 The retargeted motion of the original actor is provided to the game player by the 
image processing section, the image memory, and the image output device. At 
this time, the basic frame for main motion which will be provided, is provided to 
the game player by a frame storage section and a sliding control section so that the 
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game player can easily follow the motion of the original actor. The motion of the 
game player is stored in the game player motion data storage section real-timely. 
The motion evaluation section then evaluates the jnotion of the game player after 
comparing the motion of the game player with the retargeted motion data of the 
5 original actor. Finally, the result is provided to the game player by the image 
output device. 

To solve problems which may be occurred due to the body size difference 
between the original actor and the game player prior to analyzing the motion, the 
retargeting must be performed. That is to say, the retargeting is a processing 
10 to remove the body size difference. 

The processing sequence of the retargeting will be explained with reference 
to FIG. 3. 

Firstly, two data with respect to body sizes of the original actor and game 
player are received by input from a user or image capture of the camera using an 

15 image processing. The two data are analyzed, and the motion data of the 
~ original actor is converted so as to be appropriate to the body size of the game 
* player. In general, the three-dimensional motion data of the original actor is 
magnified or reduced based on the specific part (mostly, waist) of the 
three-dimensional motion data of game player and then the center of magnified or 

20 reduced the three-dimensional motion data of the original actor is moved. After 
moving the center, finally, the retargeted motion data of the original actor may be 
obtained by performing conversion to satisfy a restriction condition. The reason 
why the conversion is required to satisfy the restriction condition is that foot is 
floated in the air, if a three-dimensional motion data of a basketball player with 

25 height of 2m is reduced based on his waist with respect to an elementary pupil with 
height of lm. Therefore, the restriction condition that "the foot is necessarily 
touched to the ground" is required, and the three-dimensional motion data of the 
basketball player must be converted considering this restrictive condition. 
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FIG. 4 is a frame structure diagram for describing one example of an image 
frame configuration of a three-dimensional original actor according to the present 
invention. 

In FIG. 4, each of the boxes designates a motion frame stored to unit time 
5 intervals and each of the numbers on the boxes shows a frame number. It should 
be understood that the image frame of an original actor consists of a start frame, an 
end frame and motion frames. The motion frames are divided into basic motion 
frames and common motion frames. A special motion (namely, main motion of 
dance actions) of these motion frames is defined as the basic motion frame. The 

10 information with respect to the basic motion frame may be marked using a 
management program marking "a specific pose" by an expert (content maker) after 
a point of time creating a motion data or storing the motion data. The 
information with respect to the basic motion frame may be included in a portion of 
the motion frame displaying the motion of the original actor or be used as an 

15 additional file. 

The motion frame is generally stored in the form of file to display a motion 
image such as for example, a BVH file. The format of the BVH file firstly 
describes about a triple structure consisting of a several sensors for designating 
each of the parts of the body, and then displays formations on an angle and a 

20 coordinate value versus time of the sensors corresponding to each of the parts of 
the body. The coordinate versus time may be understood by this BVH file. 

FIG. 5 and FIG. 6 are description drawings for describing a method of 
displaying preparation action of an original actor. In FIG. 5, (a) shows an 
original actor motion frame displaying the motion of the original actor, (b) is a 

25 basic motion frame displaying an information on the basic motion of the motion 
frame of the original actor, and (c) shows a game player motion frame storing the 
motion of a game player following the motion of the original actor, respectively. 
As shown in FIG. 5(b), it will be understood that 5th, 10th, 17th, 19th, 21st, 25th, 
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27th, 28th, 31st, 37th, 38th, 40th, 42nd and 48th frames are designated as the basic . 
frames. As present time t is the point of time when the game player must follow 
the motion of the original actor, the real-time motion of the original actor is 
provided. Furthermore, At is to display time interval for detecting the basic 
5 motion frame. According to FIG. 5, At shows the state that 6th frame is ruled of 
the time displayed. Furthermore, according to FIG. 5, the 16th motion frame of 
the original actor is provided to the game player at the present time t, and the basic * 
motion frames (17th, 19th and 21st frames) between t and t + At are 
simultaneously displayed, for showing beforehand to the game player. 
10 For example, FIG. 6 is description drawing for describing a method of 

displaying such points with a display device. As shown in FIG. 6 (a), continuous 
motions (the 16th frame in FIG. 5) of the original actor in which a present game 
player must follow, are displayed on one region 61 of a screen, and main motions 
(17th, 19th and 21st frames in FIG.5) in which the game player will follow at next 
15 stage are sequentially and real-timely displayed according to time on the other 
regions 62, 63 and 64 of the screen. By using this display method, the game 
player identifies beforehand next main motions, so that he can easily follow the 
motions of the original actor. 

As shown in FIG. 6 (a), when setting a coordinate on the screen based on a 
20 coordinate axis, in case that the eye height He of the game player is longer than 
distance Dy between the foot position of the game player and the center position 
on the screen of the display device, a coordinate value Yi on Y axis starting a 
guide action and a coordinate value Yf on Y axis of the screen displaying 
real-timely continuous actions must satisfy following equation (1). 

25 

Yi > Yj : • • • equation (1) 

This is to further improve the property of a three-dimensional image by 

9 
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displaying the guide action in view of the eye height of the game player. 

Similarly, in case that the eye height He of the game player is shorter than 
distance Dy between the foot position of the game player and the center position 
on the screen of the display device, the coordinate value Yi on Y axis starting the 
5 guide action and the coordinate value Yf on Y axis of the screen displaying 
real-timely continuous actions must satisfy following equation (2). 

Yi < Yj • equation (2) 

10 At this time, the direction of a vector consisting of starting points Xi, Yi 

and Zi and end points Xf, Yf and Zf can be easily determined by analyzing motion 
data such as B VH of the original actor including the position values of the sensor 
attached on the articulation parts of the original actor. Therefore, when 
displaying the guide action in view of the direction of the vector, it is possible to 

15 display the guide action more actually. Furthermore, it is preferable to locate 
position points Xf, Yf and Zf on the screen displaying the continuous actions of the 
original actor on the center of the screen, for providing the game player with 
facility. Moreover, it is preferable to appropriately locate start points Xi, Yi and 
Zi in view of the number of preparation action frame which will be displayed and 

20 horizontal size of the screen. The eye height He of the game player can be easily 
calculated using statistical value from the height of the game player to his eye 
position. Here, the height of the game player is input by the game player or 
operated by a camera. 

FIG. 7 is a flow chart showing overall action displaying and evaluating the 

25 motion using the motion frame of FIG. 4. A motion data, a music DB and a 
character DB are selected from an input section by the game player (step S71). 
Next, the body size of an original actor is compared with that of the game player 
using the body size inputted by the game player or obtained by analyzing the 
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image of the camera (step S72). A three-dimensional image coordinate value 
data of the original actor is converted using this comparison value (step S73). At 
this time, since the number and position of sensors attached on the game player are 
different from the number and position of sensors attached on the original actor for 
5 capturing the motions of the original actor, it is necessary to primarily agree with 
targets which will be compared for the purpose of evaluating the targets 
appropriately. Accordingly, the position data of the reference sensor is calculated 
from the motion data of the original actor based on the number and position of the 
sensors attached on the game player (step S74). 
10 Thereafter, the three-dimensional motion data of the original actor is 

real-timely displayed, and the guide action is simultaneously displayed (step 75). 
While the three-dimensional motion of the original actor may be displayed prior to 
retargeting the motion of the original actor, it is preferable to display the 
three-dimensional motion of the original actor after retargeting. In case of 
15 displaying the three-dimensional motion of the original actor prior to retargeting, 
~ while it is possible to substantially display the motion of the original actor, it is 
• difficult to exactly evaluate the motions of the game player and original actor. In 
case of displaying the three-dimensional motion of the retargeted original actor, it 
is possible to easily compare the motion of the game player with the motion of the 
20 original actor. Namely, in case of displaying the motion of the game player by 
overlapping the motion of the original actor with the motion of the game player, 
there is advantage that the game player may easily identify how different from the 
motion of the original actor by using the retargeted motion. 

The motion of the game player following the motion of the original actor 
25 displayed on the screen is stored real-timely by the frame (step S76). At this time, 
whether or not the three-dimensional motion frame of the original actor displayed 
at time t, is the basic motion frame, is determined (step 77). In case of the basic 
motion frame, the motion frame of the game player identified real-timely is 

11 
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compared with the position data of the reference sensor calculated at the step 74 
and evaluated (step S78). The steps S76 and S77 may be performed regardless of 
sequence. The step S74 may be also performed after step S77. Namely, 
whether or not the three-dimensional motion frame of the original actor is the basic 
5 motion frame, is determined, and with calculating the position data of the reference 
sensor for the basic motion frame, the calculated resultant value can be compared 
with the sensor position obtained from the motion of the game player and 
evaluated. When either the step S76 or the step S74 is performed after the step 
S77, only the motion frame of the game player following the basic motion frame of 
10 the original actor is stored, so that only the position data of the reference sensor 
will be operated. 

Thereafter, whether or not the frame is the last frame, is determined (step 
S79). In case of the last frame, the step S75 is again started, while in case of not 
being the last frame, the final result is displayed using the resultant value of the 

15 comparison and evaluation (step S80). 

Since it must be performed real-timely within a short time to compare and 
evaluate whether or not the motion of the game player is agreed with the basic 
motion frame of the original actor, it is performed by procedure proposed in FIG. 8 
for the purpose of reducing time required for an image evaluation procedure. 

20 1. The step of operating the position value of the reference sensor (step 

S81) 

As above described, to appropriately evaluate targets in the number and 
position of sensors attached on the original actor for capturing his the motion and 
attached on the game player, it is necessary to primarily agree with the comparison 
25 targets. Accordingly, the three-dimensional position value of the reference sensor 
is operated from the reference motion data of the original actor based on the 
number and position of the sensors attached on the game player. This is defined 
as the position value of the reference sensor. It is assumed that the position value 
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of the reference sensor for a sensor 1 attached on the game player is (XI, Y1,Z1), 
for facilitating description. 

2. The step of calculating two-dimensional coordinate values 
corresponding to each of the cameras (step S82) 

5 As above described, a plurality of cameras are used for identifying the 

three-dimensional positions of the sensors attached on the game player. The 
three-dimensional image is obtained by synthesizing the two-dimensional image 
obtained from the plurality of the cameras. 

By inversely performing procedure the same as jnethod obtaining the 

10 three-dimensional image from the two-dimensional image, the two-dimensional 
coordinate value of corresponding to sensor position included in each of the 
cameras for embodying corresponding to the three-dimension image of the original 
actor, can be calculated. A two-dimensional coordinate value (XI 1, Yll) 
displayed on a first camera, a two-dimensional coordinate value (X21, Y21) 

15 displayed on a second camera, a two-dimensional coordinate value (Xnl, Ynl) 
displayed on a nth camera, can be respectively calculated from the reference 
position value (XI, Yl, Zl) of the step S81 by this image processing method. 

3. The step of calculating a target region of each of cameras (step S83) 

A certain region is set at a exact position where a sensor is discovered by 
20 each of the cameras. For example, when a game player exactly follows the 
motion of an original actor, the exact position value of the sensor 1 discovered by 
the camera 1 must be (XI 1, Yl 1). However, it is impossible that the game player 
exactly follows the motion without a selected error. Accordingly, when setting a 
certain region having for example, margin of 5 based on (XI 1, Yll), a square 
25 region consisting of (Xll - 5, Yll - 5), (Xll + 5, Yll - 5), (Xll + 5, Yll + 5), 
and (XI 1 - 5, Yl 1 + 5) is selected. The selected region is referred as a target 
region for facility. 

4. The step of checking whether or not the sensor is detected in the target 
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region (step S84) 

Whether or not the image coordinate of the sensor attached on the game 
player is introduced in the target region set at step S83 among the image data of 
each of the cameras is checked. As a conventional image process method, in case 
5 using the method that overall region of image data outputted from each of the 
cameras is checked, the position value of the sensor attached on the game.player is 
identified, and compared with the sensor position value of the original actor, long 
time is required for the image processing. Accordingly, the target region where 
the sensor will be discovered is set beforehand. Furthermore, by only checking 

10 not the overall region but a certain target region, image processing time can be 
considerably reduced. When checking the certain region in this manner, it can 
be easily evaluated that in case the sensor is discovered in the target region of all 
the cameras, the motion of the game player is the same as motion of the original 
actor, while in case the sensor is not discovered, the motion of the game player is 

15 different from the motion of the original actor. Furthermore, the 
three-dimensional position information can be obtained by combining the 
two-dimensional data detected in this manner. Namely,* a method that the 
three-dimensional information which will be finally used, is calculated by the 
two-dimensional detection information obtained from at least two cameras based 

20 on one sensor, and the three-dimensional data of the original actor is then 
compared with the calculated information, can be used. Since an algorithm in 
which a three-dimensional information is obtained from a plurality of 
two-dimensional information, follows a conventional theory of a computer vision 
field, the description of the above algorithm is omitted here. 

25 

. FIG. 9 is a flow chart showing performance sequence of the present 
invention in detail. All the body action of an user can be pursued using music in 
accordance with selection of the user and high performance motion capture system 
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(steps S101 and SI 02). A character and stage which can be applied to the case 
viewing this apart from the image of the original actor are then selected (steps 
SI 03 and SI 04). Next, the motion data of the original actor is read from his the 
motion capture DB (step S105), and a character image is overlaid on the motion 
5 data of the original actor (step S106). Since the size of the image file increases 
when using the image of the original actor according to that, the amount of data 
processing can be reduced by overlaying the character image stored at the motion 
coordinate of the original actor beforehand. Next, an operation matching the 
motion data with the music data is performed (step SI 07). A display device is 

10 used for the motion data of the original actor and a music is outputted by a sound 
device (step S108). The user takes motion according to the output data (step 
SI 09). Thereafter, all the body motions are captured by a plurality of cameras, a 
mark is identified from the image, and the motion is pursued by the frame (steps 
SI 10 and Sill). The two-dimensional coordinate value is extracted using the 

15 image data obtained by this method, and the three-dimensional motion coordinate 
value is obtained from the extracted data (steps SI 13 and S 1 14) 

Next, using the coordinate data of the original actor and the* 
three-dimensional coordinate value of the learner motion -captured, the body size 
of the original actor is compared with that of the learner (steps SI 15 and SI 16). 

20 When the two body sizes are different from each other in real comparative 
resultant, retargeting is performed (steps SI 17 and SI 18). The retargeting can be 
performed during the motion-capture of the learner in accordance with flow as 
shown in FIG. 9, or prior to carrying out the motion-capture. After retargeting, 
in case of the basic frame, whether or not the data of the original are agreed with 

25 the data of the learner is checked by comparing with the two data (steps SI 19 
and SI 20). When the two data are agreed with each other, a score is given, and 
the total time agreed is counted, thereby displaying advertisement objects obtained 
using action agreement message or advertisement object production DB (steps 
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SI 22 to S124). A motion capture data is stored in a user motion storage section, 
and then whether or not the frame is a last frame is checked (steps SI 25 and SI 26). 
Thereafter, while in case of not being the last frame, next motion is captured, in 
case of the last frame, the game is over. 
5 When the motion data of the original actor is not agreed with the motion 

data of the learner, after checking whether or not a function which the game is over 
in accordance with disagreement time is set (step SI 27), in case that the two data 
are not agreed with each other more than certain times, the game is over 
intentionally. 

10 As above described, according to the present invention, the motion of a 

game player following the motion of a dancer can be real-timely evaluated by a 
motion game apparatus. 

Furthermore, a method playing a motion data is proposed so that the game 
player can easily follow the motion of the original actor in the motion game 

15 apparatus displaying the motion of the original actor, thereby improving facility of 
user. Moreover, the motion of the game player following the motion of the 
original actor is real-timely and rapidly evaluated, thereby enabling the motion 
game apparatus to be utilized. 

The invention may be embodied in other specific forms without departing 

20 from the spirit or essential characteristics thereof. The present embodiments are 
therefore to be considered in all respects as illustrative and not restrictive, the 
scope of the invention being indicated by the appended claims rather than by the 
foregoing description and all changes which come within the meaning the range of 
equivalency of the claims are therefore intended to be embraced therein. 

25 
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CLAIM: 

What is claimed is: 

5 1. A method of displaying a motion data in a motion game apparatus having 
information of a basic frame displaying main motion of an original actor, and 
playing the motion data of the original actor consisting of a plurality of frames 
on a display device, comprising: 

a first step of setting play position points Xf, Yf and Zf on the display 
10 device for playing continuous action of the original actor and initial position 
points Xi, Yi and Zi on the display device for displaying beforehand the basic 
frame which will be played after a predetermined time At; 

a second step of displaying continuously the continuous action of the 
original actor at the play position points Xf, Yf and Zf based on present time t, 
15 and simultaneously displaying the basic frame at the initial position points Xi, 
Yi and Zi after drawing out the basic frame which will be displayed at t+ At; 
and 

a third step of displaying with gradually moving the basic frame 
displayed at the initial position points Xi, Yi and Zi to the play position points 
20 Xf , Yf and Zf as time goes by. 

2. The method according to claim 1, wherein the image size of the basic frame 
displayed at the initial position points Xi, Yi and Zi is smaller than the image size 
for playing the continuous action of the, original actor displayed at the play position 
25 points Xf, Yf and Zf, and wherein the movement at the third step is magnification 
movement. 

17 
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3. The method according to claim 1, wherein the play position points Xf, Yf and 
Zf are set to the center of the screen of the display device, and wherein the initial 
position points Xi, Yi and Zi are changed up and down based on the level of the 
display device in accordance with the eye height of a game player following the 

5 motion of the original actor. 

4. A method of displaying a motion data in a motion game apparatus having 
information of a basic frame displaying main motion of an original actor, 
consisting of a plurality of frames, having play position points Xf, Yf and Zf on a 

10 display device for playing continuous action of the original actor and initial 
position points Xi, Yi and Zi on the display device for displaying beforehand the 
basic frame which will be played after a predetermined time At, and displaying the 
motion data of the original actor on the display device, comprising: 

a first step of displaying continuously the continuous action of the original 

15 actor at the play position points Xf, Yf and Zf based on a present time t and 
simultaneously displaying the basic frame at the initial position points Xi, Yi and 
Zi after drawing out the basic frame which will be displayed at t + At; and 

a second step of displaying with gradually moving the basic frame 
displayed at the initial position points Xi, Yi and Zi to the play position points Xf, 

20 Yf and Zf as time goes by. 

5. The method according to claim 4, wherein the image size of the basic frame 
displayed at the initial position points Xi, Yi and Zi is smaller than the image size 
for playing the continuous action of the original actor displayed at the play position 

25 points Xf, Yf and Zf, and wherein the movement at the second step is 
magnification movement, 

6. A method of evaluating the motion of a game player following the motion data 
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of an original actor displayed on a display device in a motion game apparatus 
having information of a basic frame displaying main motion of an original actor, 
consisting of a plurality of frames, and setting play position points Xf, Yf and Zf 
on the display device for playing continuous action of the original actor and initial 

5 position points Xi, Yi and Zi on the display device for displaying beforehand the 
basic frame which will be played after a predetermined time At, and comprising: 

a first step of storing a three-dimensional motion data of the original actor 
retargeted after converting the three-dimensional motion data of the original actor 
by reflecting body size of the game player; 

10 a second step of displaying continuously the continuous action of the 

original actor at the play position points Xf, Yf and Zf based on present time t 
using the three-dimensional motion data of the retargeted original actor and 
simultaneously displaying the basic frame at the initial position points Xi, Yi and 
Zi after drawing out the basic frame which will be displayed at t + At; 

15 a third step of displaying with gradually moving the basic frame displayed 

at the initial position points Xi, Yi and Zi to the play position points Xf, Yf and Zf 
as time goes by, 

a fourth step of extracting and storing the three-dimensional motion data of 
the game player learning the same motion with following the motion of the 
20 original actor displayed; 

a fifth step of calculating a coordinate value of a reference sensor for the 
original actor corresponding to a sensor position attached to the game player from 
the basic frame displayed at the play position points Xf, Yf and Zf of the third step; 
and 

25 a sixth step of comparing the three-dimensional motion data of the game 

player with the coordinate value of the reference sensor. 

7. The method according to claim 6, wherein the fourth and fifth steps are 

19 
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performed regardless of sequence, and wherein the fifth step is performed 
between the first and sixth steps. 

8. A method of evaluating a motion of a game player following a motion data of 
5 an original actor displayed on a display device in a motion game apparatus having 

information of a basic frame displaying main motion of an original actor, 

' consisting of a plurality of frames, setting play position points Xf, Yf and Zf on the 

display device for playing continuous action of the original actor and initial 

position points Xi, Yi and Zi on the display device for displaying beforehand a 
10 basic frame which will be played after a predetermined time At, and having a 

plurality of optical sensors attached to the game player, comprising: 

a first step of storing a three-dimensional motion data of the original actor 

retargeted after converting the three-dimensional motion data of the original actor 

by reflecting body size of the game player; 
15 a second step of calculating positions of the plurality of optical sensors 

attached to the game player from the three motion data of the retargeted original 

actor; 

a third step of displaying continuously the continuous action of the original 
actor at the play position points Xf, Yf and Zf based on a present time t using the 
20 three-dimensional motion data of the retargeted original actor and simultaneously 
displaying the basic frame at the initial position points Xi, Yi and Zi after drawing 
out the basic frame which will be displayed at t+ At; 

a fourth step of displaying with gradually moving the basic frame 
displayed at the initial position points Xi, Yi and Zi to the play position points Xf, 
25 Yf and Zf as time goes by; 

a fifth step of extracting and storing the position of the optical sensor of the 
game player learning same motion with following the motion of the original actor 
displayed; and 

20 
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a sixth step of comparing the position of the sensor calculated from the 
basic frame displayed at the play position points Xf, Yf and Zf of the fourth step 
with the position of the sensor of the game player stored at the fifth step. 

5 9. A method of evaluating a motion of a game player following a motion data of 
an original actor with displaying the motion data in a motion game apparatus 
having a plurality of camera for monitoring a plurality of sensors attached the body 
of the game player, comprising: 

a first step of calculating a position value of a reference sensor detected 
10 when the game player exactly follows the motion of the original actor from the 
motion data of the original actor; 

a second step of calculating the position values of the sensors 
corresponding to each of the cameras, the sensors being installed to each of the 
cameras for accomplishing the position value of the reference sensor; 
15 - a third step of setting a target region corresponding to each of the cameras 

- having a certain region range based on the position values of the • sensors 
corresponding to each of the cameras; and 

a fourth step of determining whether or not the sensor attached on the game 
player is detected at the target region. 

20 

10. The method according to claim 9, wherein the position value of the 
reference senor is a three-dimensional coordinate value, and wherein the position 
values of the sensors corresponding to each of the cameras are two-dimensional 
25 coordinate values. 
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